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Curricular analysis indicates that the U.S. students are introduced to multiplication in additive 
terms (as the replication of equal groups and repeated addition). But the virtue of this 
introduction for supporting students’ understanding of the full range of multiplicative 
relationships is unclear. This paper reports an analysis of all grade 4 released NAEP items that 
expressed a multiplicative relationship, focusing on the range of relationship types, related 
quantities, and item difficulty. Results show that multiplicative items (1) frequently presented 
discrete quantities and equal group situations, (2) were easier when they involved discrete 
versus continuous quantities, and (3) within discrete items, equal groups and array items were 
easier than other types. These results provide measured support for the conjecture the additive 
introduction to multiplication may limit the development of elementary students’ understandings. 
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Objectives of the Study 

Can analyses of released items from the National Assessment of Educational Progress 
(NAEP)—the “nation’s report card”—and performance on those items shed light on questions of 
interest to mathematics educators, beyond what has been reported in summary volumes (e.g., 
Klosterman & Lester, 2007)? This analysis addressed that question for the broad content area of 
multiplicative relationships. “Multiplicative relationships” designates a set of tasks and 
situations, numerical and quantitative, that engage students in multiplicative reasoning and in 
carrying out numerical operations of multiplication or division. Given the introduction of 
multiplication and multiplicative relationships in grades 2 and 3 in U.S. classrooms, the analysis 
examined released items from grade 4. 

Multiplicative relationships are significantly more diverse and challenging for students to 
master than additive relationships (Nunes & Bryant, 1996; Vergnaud, 1983, 1988). Students 
work on multiplicative relationships for many years, and this work is intensive in the upper 
elementary through middle school years. They are introduced to the operations of multiplication 
and division and some “applied” situations in grades 2 and 3. In the U.S., this introduction is 
essentially additive in nature. Quantitatively, multiplication is presented as involving the 
replication of equal-sized groups of discrete objects; numerically, it is presented as repeated 
addition (Smith, 2017). Research on students’ understanding of different multiplicative 
relationships raises questions about whether this instructional foundation effectively supports 
students’ access to and understanding of the full range of multiplicative relationships. This 
analysis of released NAEP items is one small step in a larger effort to address that critical 
question. The NAEP analysis complements the results of many prior studies that have assessed 
students’ understanding of multiplicative relationships (as summarized in Greer [1992] and Harel 
& Confrey [1994]). 

Two main questions focused the analysis: (1) what multiplicative situations (type and 
frequency) appear in released Grade 4 items and (2) how well do the performance results align 
with the additive introduction to multiplication? For example, are items presenting the 
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replication of equal groups easier than other types of situations that are less amenable to 
replication and repeated addition? 


Theoretical Perspective 

Broadly, the larger inquiry that motivated this analysis was framed in constructivist terms. If 
learning is a social and psychological process of adapting prior understandings to cope with new 
and problematic mathematical situations (e.g., Wood, Cobb, Yackel, & Dillon, 1993), then how 
we introduce students to multiplicative relationships matters greatly for their subsequent work to 
engage the full multiplicative conceptual field (Vergnaud, 1983). Understanding multiplication 
and division means knowing where and why situations encountered in the world are 
“multiplicative,” not additive. Mastery of the numerical aspects multiplication and division (1.e., 
basic facts, algorithms, and properties of operations) may contribute to understanding 
multiplicative relationships but is neither sufficient nor central. 

A framework of types of multiplicative reasoning and quantitative situations that typically 
elicit such reasoning framed the analysis of the released items. Quantities are countable or 
measureable attributes of objects or collections of objects that are constituents of situations that 
student encounter and reason about in school and the everyday world (Smith & Thompson, 
2008). To use their mathematical knowledge effectively in reasoning about and resolving these 
situations, students must consider the quantities involved and how they are related. Table 1 
presents (and relates) different types of multiplicative reasoning and types of situations. But 
situations do not determine students’ reasoning about them. Rather, the correspondence below 
reflects how prior research has characterized situations in relation to multiplicative reasoning. 
“Replication,” an additive form of reasoning, has been included for “coverage” of the released 
items and because U.S. curricula treat replication as multiplicative. 


Table 1: Types of Multiplicative Reasoning and Situations 


Types of Multiplicative Reasoning | Types of Quantitative Situations 

Replication Equal groups; rectangular arrays; 
some area and volume situations 

One-to-many Unit conversion 

Scaling Comparison; price (discrete); 
rate/cost (continuous) 

Successive partitioning Folding; splitting 

Composition Cartesian product (discrete); area; 
volume (continuous) 


Central to the analysis of situation types is the distinction between discrete and continuous 
quantities. Discrete quantities are sets of objects; their numerical value can be determined by 
counting. Continuous quantities are initially attributes of unsegmented objects (e.g., distances or 
lengths, areas, time periods between two events). Their measurement requires the selection and 
iteration of a unit (a smaller piece of the target quantity). Their numerical value is the number of 
such units that collectively fill up or “exhaust” the initial quantity. 

Rectangular arrays and some area and volume situations are listed in Table 1 along with 
equal groups because (1) array situations support replication reasoning (when a row or column of 
objects is interpreted as a group) and (2) area and volume situations are often presented as arrays 
of squares or stacks of cubes that similarly support replication. Price indicates situations where 
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some discrete number of items has been purchased. It is related to, but a special case of the more 
general set of rate/cost situations that accept measured quantities (e.g., “9.45 gallons of gas”). 
Cartesian product and area/volume are both multiplicative compositions, where the product 
differs from both factor quantities. In the former, “pants” and “shirts” are different quantities 
than “outfits.” Similarly, both rectangular area and prism volume are different quantities than the 
lengths from which they are composed. 

The entries in Table 1 suggest that understanding multiplication involves grasping 
fundamentally different forms of multiplicative relationship that many students may see as 
conceptually distinct, especially early in their mathematical experience. 


Methods 

Items from nine Mathematics assessments (administered in 1990, 1992, 1996, 2003, 2005, 
2007, 2009, 2011, and 2013) have been released for public examination 
(https://nces.ed.gov/nationsreportcard/nqt/). NAEP characterizes items by content area—Number 
properties and operations (NPO), measurement (M), geometry (G), data, statistics and 
probability (DSP), and algebra (A) and by format—multiple-choice, short constructed response, 
and extended constructed response. Released NAEP items are characterized by difficulty, as 
“easy” (performance > 60% correct), “medium” (performance is between 40% and 59% correct), 
or “hard” (performance < 40% correct). 

All 388 released grade 4 items were examined and coded by the author as either additive, 
multiplicative, or other. Multiplicative items presented one of the situation types listed above in 
Table 1 (including replication). Additive items presented one of three types of additive 
relationship—combine, separate, or compare. Additive items included area and volume/capacity 
items that presented collections of squares and cubes that supported counting and (additive) 
comparison. Other items presented content topics such as place value, ordering, estimating, 
fractions, graphing, and stating probability, where neither an additive nor multiplicative 
relationship between numbers or quantities was expressed. 

Multiplicative items were found in all five content areas but were most common in NPO and 
M domains. All multiplicative items were first coded as “numerical” or “quantitative.” 
Numerical items presented written numerals and operations with minimal prose. “Quantitative” 
items were primarily expressed in words, where numerals were associated with quantities. Some 
items were presented entirely in prose (as “word problems”); others presented tables or figures 
with the written text, and the tabular or figural information was necessary for solving the item. 
Quantitative items were then coded for the type of quantities involved, discrete or continuous. 
Discrete items were further distinguished according to the type of relationship presented. Eight 
types proved sufficient for coding all discrete items: Equal groups, equal shares, rectangular 
arrays, money, price, multiplicative comparison, unit conversion, and Cartesian product. 
Continuous items presented length, area, or volume/capacity measurement situations where the 
quantity could not be evaluated with additive reasoning. Four types were sufficient for coding 
continuous items: Unit conversion, equal shares, multiplicative comparison, and computation. 


Results 
For a broad overview, Table 2 presents an overview of all 388 released grade 4 items by 
year. Column 3 lists the total number of multiplicative items; column 4 lists the total number of 
additive items; and columns 5-8 characterize the multiplicative items, first as numerical or 
quantitative items and then for quantitative items, as discrete and continuous. 
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Table 2: General Character of Grade 4 Items (1990-2013) 


Year | All | Mult | Add M; M; M, Quan; M, Quan; 
Number | Quan Discrete Continuous 
2013 46 14 9 6 8 6 2 
2011 49 20 14 7 13 10 3 
2009 31 6 8 1 5 3 2 
2007 54 14 12 3 11. 9 2 
2005 32 5 11 2 3 3 0 
2003 59 23 9 3 20 16 4 
1996 25 9 6 1 8 6 2 
1992 59 20 13 6 14 12 2 
1990 33 10 - 6 4 3 1 
Total 388 | 121 89 35 86 68 18 


The number of released items and the number of multiplicative items varied substantially across 
the nine assessments, but multiplicative items were generally more frequent than additive items. 
In most years, multiplicative quantitative items outnumbered multiplicative numerical items, 
often dramatically. Among quantitative items, those presenting discrete quantities were at least 
twice as frequent as those presenting continuous quantities. 

More substantively, the distribution of discrete items across the eight types listed above was 
not uniform (Table 3). 


Table 3: Frequency of Grade 4 Discrete Multiplicative Items by Type (1990-2013) 


Year | Disc | Grps | Share | Array | Comp | Unit | Money | Price | C.P. 
2013 6 2 0 0 2 1 0 0 1 
2011 10 3 2 0 2 3 0 0 0 
2009 3 1 0 0 0 1 1 0 0 
2007 9 3 0 0 i 3 1 1 0 
2005 3 2 0 0 1 0 0 0 0 
2003 16 5 2. 1 1 1 0 5 1 
1996 6 2 0 2 0 1 0 1 0 
1992 12 5 0 0 0 0 1 5 1 
1990 3 0 0 1 0 0 0 2 0 
Total | 68 23 4 4 7 10 3 14 3 


oN 


Note: “Grps” = equal groups, 
Cartesian product 


‘Comp” = comparison, “Unit” = unit conversion, ““C.P.” = 
+ | . | 


Equal groups was by far the most frequent type of situation presenting discrete quantities in 
multiplicative relationship, followed by price and unit conversion. Equal groups and unit 
conversion items always involved quantities with whole number values, where price items 
involved some whole number of items at a cost represented as decimal (e.g., $0.87 or $2.79). 

Many authors have argued that multiplicative relationships are intrinsically more difficult for 
students to master than additive relationships (e.g., Vergnaud, 1983, 1988). Is this claim reflected 
in the NEAP results? Overall, the entries in Table 4 indicate an affirmative answer. 
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Table 4: Relative Difficulty of Grade 4 Additive and Multiplicative Items (1990-2013) 


Year | All | Mult. | Add. | Avg % Corr;A | Avg % Corr;M | %A-—%M 
2013 46 14 9 46.8 43.6 32) 
2011 49 20 14 58.2 48.8 9.4 
2009 oi 6 8 57.4 52.8 4.6 
2007 54 14 12 61.3 46.0 15.3 
2005 32 5 11 53.3 58.2 -4.9 
2003 59 23 9 57.8 50.8 7.0 
1996 25 9 6 54.2 42.0 12.2 
1992 59 20 13 45.2 40.4 4.8 
1990 33 10 | 44.6 44.2 0.4 
Total | 388 | 121 89 


On seven of nine assessments, multiplicative items were more difficult than additive items, and 
in three (1996, 2007, and 2011) significantly so. The difference was negligible in 1990 and had 
the opposite sign in 2005. 

But item difficulty across types of multiplicative relationships was the central focus of this 
analysis. Table 5 below reports percent correct for items presenting each type of discrete 
multiplicative relationship. The first value in right-most column lists the average percent correct 
for all items of that type. The values in parentheses are the average percent correct for a 
meaningful subset of those items, as explained below. The other columns list the number of 
items rated “easy,” “medium,” and “hard” and the percent correct for each item in those three 


categories. 

Table 5: Difficulty of Grade 4 Discrete Multiplicative Items by Type (1990-2013) 
Discrete N N % N % N % Avg % 
sub-type Easy Med Hard 
Equal 23 4 75,61, 80, | 11 | 50, 53, 59, 53, 8 | 38, 35, 23, 49.0 
groups 70 56, 46, 57, 50, 21, 36, 39, (56.4) 

55, 47, 48 Sit 
Price 14 2 70, 62 4 | 58, 58, 53, 48 8 | 4.35.39; 31, | 39:3 
17, 8, 9, 21 (53.2) 
Unit 10 6 66, 75,65, | 2 | 53,44 an |S i 58.3 
Conversion 85, 78, 61 (62.9) 
Compare a 1 72 2 | 47,47 4 | 32, 34, 24, 43.0 
34 (36.3) 
Array 4 1 79 2 | 50, 48 1 35 53.0 
Money 5 1 60 1 | 58 1 | 20 46.0 
Equal shares | 4 2 47, 51 2 23; 38 39.8 
Cartesian 3 1 48 2 24; 28 33.3 
product 
Total 68 | 15 25 28 45.9 


Items in three discrete types were somewhat easier overall (Unit Conversion, Array, and 
Equal Groups, in descending order), with average correct at or slightly above 50%. Six of the 10 
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Unit Conversion items provided the conversion ratio explicitly in the item (e.g., 1 qt. = 2 cups). 
The two “hard” conversion items (39% and 17% correct) both presented 3:1 ratios, where the 
other eight items presented conversion ratios of 2:1, 5:1, 10:1, or 100:1. Arrays were either 
represented directly in diagram (n = 1) or described in words (n = 3). The “easy” item (79% 
correct) described a particularly familiar array—two rows of six cookies on a cookie sheet. 

As shown, Equal Groups and Price items were often difficult; eight items of both types were 
“hard.” But within both types, hard items often involved two or more steps, where a 
multiplicative relationship was involved in at least one step. For example, some two-step Price 
items asked for the change received for the purchase of a set of items at a given price when a 
specific bill was given for payment—requiring both multiplicative and additive reasoning. Some 
two-step Equal Groups items introduced more than one group (e.g., students in a class and buses 
with maximum capacity for students). Most multi-step items were more difficult than single step 
items of the same type. The average percent correct for the thirteen single-step Equal Groups 
items was 56.4% (as shown), where the corresponding average for the ten multi-step items was 
39.2%. Similarly, the six single-step Price items were considerably easier (average 53.2% 
correct) than the eight multi-step items (average 28.9% correct). 

By contrast, Compare, Equal Shares, and Cartesian Product items were more challenging, at 
43%, 39.8%, and 33.3% average correct, respectively. The majority of Compare and Cartesian 
Product items were “hard,” even when six of the seven Compare items involved a 2:1 ratio. Only 
one, presenting ten stars and five triangles in a 3 by 5 array and four possible ratios, was “easy” 
(72% correct). Without that item, average correct fell to 36.3%. Of the four Equal Shares items, 
none called for simply distributing some discrete quantity equally to a given number of 
recipients. One “hard” item (38% correct) asked students to distribute 24 wheels to bikes and 
wagons in two different ways; one “medium” item (47% correct) required interpreting the 
remainder after equal sharing. Cartesian Product items were difficult even though support was 
provided for solving two of the three (i.e., items provided the solution for smaller factors). 

Finally, Table 6 presents the performance on the N = 18 multiplicative items that presented 
length, area, or volume/capacity quantities and within each quantity, the type of multiplicative 
relationship involved. 


Table 6: Difficulty of Grade 4 Continuous Multiplicative Items by Type (1990-2013) 


Continuous N N % N Med | % N % Avg 
Sub-type Easy Hard % 

Length 7 1 6 30.6 
compare 2 21533 30.0 
equal shares 4 47 27, 26, 23 30.8 
unit convert 1 31 31 

Area 6 1 1 4 36.5 
compute rect 4 23, 24, 24, 19 22.5 
compare 2 78 51 64.5 
Volume/capacity | 5 2 1 2 47.4 
unit convert 3 67, 61 32 53.3 
compute stack 1 56 56 
compare 1 21 21 

Total 18 3 3 12 37.2 
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Overall, the results show that multiplicative items presenting continuous quantities were 
generally more difficult than those with discrete quantities. Six of seven length items were 
“hard,” as were four of six area items—even when the area items involved a single-step. 
However, the two items comparing areas— both presenting sectors of partitioned circles—were 
markedly easier (average 64.5% correct) than the corresponding discrete comparison items (n = 
7; average 43.0% correct). The three volume/capacity unit conversion items were somewhat 
more difficult (average 53.3% correct) than the discrete unit conversion items (n = 10; average 
58.3% correct). As was true for the discrete items, all three continuous items stated the 
conversion ratio in the item text. 


Discussion 

The analysis was revealing in two principal ways. First and generally, the careful 
examination of released NAEP items supported a finer-grained analysis of U.S. grade 4 students’ 
successes and challenges—as a proxy measure of national understanding—than the published 
volumes have thus far (Klosterman & Lester [2007] and prior volumes in that series). The 
released item set provided greater access to the items as presented to students, their difficulty, 
and the details of item performance. Second and more specific to this inquiry, the analysis 
provided a measure of empirical support for the concern that the additive introduction of 
multiplication may present challenges for the growth of students’ understanding beyond equal 
groups of discrete objects and repeated addition. However, that support was mixed and 
complicated by many factors outside the frame of the analysis. Since many factors other than 
quantity and relationship type likely contribute to item difficulty, the analysis shows the 
difficulty inherent in establishing what makes an item easy (or challenging) for students. 

Some of the results are consistent with (a) the curricular introduction of multiplication as the 
replication of equal groups of discrete quantities and repeated addition and (b) the concern that 
that an additive foundation likely makes extension to a wider set of situations problematic. 
Additive items were on average easier than multiplicative items. Second, items posing 
multiplicative relationships among discrete quantities were much more frequent and generally 
easier than item involving continuous quantities. Third, situations involving Equal Groups were 
the most frequent of type of discrete multiplicative relationship and were easier than other 
discrete item types that are less amenable to interpretation as equal groups—specifically, 
Comparison and Cartesian Product. But this was not always the case; Unit Conversion items 
were easier on average than both Equal Groups and Array items. Overall, the results are 
consistent with and do not remove the concern that the additive introduction to multiplication 
and multiplicative relationships may support initial access to multiplicative relationships, but at 
the cost of later conceptual challenges as application extends both quantitatively and numerically 
(e.g., to fractions, decimals, and negative numbers). 

There are numerous limitations to this analysis and more broadly to using NAEP released 
items to address questions about the effects of curricular approaches on student learning. Perhaps 
the most important is that the released item set stands in uncertain relationship to the larger 
corpus of items where NAEP has performance data. The conditions under which a NAEP item is 
released to the public are unknown. Also, where the type of multiplicative item and the type of 
related quantities may well influence the item difficulty, other factors do so as well, including the 
numerical values of the quantities (even within the set of whole numbers), the length and clarity 
of item prose, and the nature of support provided (e.g., stating conversion ratios or not). Third, 
multi-step problems present challenges for characterizing items as additive or multiplicative and 
for judging the sources of difficulty. Fourth, though the capacity to explain one’s reasoning may 
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be a better indicator of understanding than producing the right numerical answer, the number of 
released NAEP items requiring explanation (that is, both short and extended constructed- 
response items) have been small in number, in this target content area and likely in most others. 
The smaller the item set, the more perilous any conclusion drawn from such data becomes. Last, 
the analysis has been completed by a single person and the coding scheme must be shown to be 
reliable. 

Despite these limitations, released NAEP items are an underused resource for mathematics 
education researchers who wish to address questions of learning at a national level. Where it is 
unlikely that any similar analysis of items selected for a given topic or response type (e.g. 
constructed-response) will resolve questions about student understanding, they may contribute to 
inquiries that draw on multiple sources of evidence. NAEP released item data, reflecting such a 
large and nationally-representative sample of students, is a unique source of evidence. Its 
analysis can provide either general support (as in this case), no support, or contradictory 
evidence for a given hypothesis. 
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